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Introduction 

A  persistent  challenge  in  the  High-Perfonnance  Computing  (HPC)  community  is  how  to 
provide  remote  visualization  capability  to  its  users.  A  dynamic  and  economical  solution  is  a 
KVM-over-IP  technology,  which  uses  a  pre-existing  TCP/IP  network  to  transmit  KVM  data 
between  two  locations.  However,  the  level  of  performance  and  functionality  present  in  the 
current  consumer-level  KVM-over-IP  devices  makes  them  less  than  desirable  for  DoD  High- 
Perfonnance  Computing  applications. 

Objective 

To  address  the  specific  needs  of  the  DoD  HPC  community,  the  RDECOM-TARDEC 
HPC  Group  undertook  a  3 -year  development  effort  through  the  pursuit  of  an  Army-funded 
Phase-II  Small  Business  Innovative  Research  (SBIR)  effort  with  IP  Video  Systems  (formerly 
known  as  TeraBurst)  to  produce  a  version  of  their  V2D  product  with  advanced  features.  At  the 
time  of  this  paper’s  publication,  the  SBIR  will  be  near  completion.  Additional  testing  and  future 
demonstrations  are  expected  subsequent  to  this  paper. 

Results 

To  accommodate  remote  use  of  the  high-end  visualization  capabilities  of  a  DoD  High- 
Performance  Computing  facility,  many  advanced  features  are  necessary.  TARDEC-HPC's  SBIR 
with  IP  Video  Systems  indicated  specific  requirements  for  creating  a  KVM-over-IP  device  that 
could  be  used  for  HPC  visualization  purposes.  These  requirements  included  support  for  USB 
keyboard  and  mouse,  multi-channel  digital  audio,  full-duplex  RS232  transmissions,  and  receiver- 
side  graphic  genlock  support. 

Performance 

This  paper  discusses  the  setup,  results,  and  challenges  associated  with  a  remote  KVM 
over  IP  usage  by  testing  the  prototype  V2D  hardware.  These  field  tests  were  performed  between 
two  locations  on  separate  networks  connected  only  via  the  Internet. 

Significance  to  DoD 

As  a  result  of  this  SBIR  effort  to  help  increase  the  capabilities  of  IP  Video  System’s  V2D 
product,  providing  remote  visualization  access  to  DoD  HPC  Centers  via  KVM  over  IP 
technology  is  not  only  possible,  but  very  usable  even  with  modest  bandwidth  availability.  The 
use  of  this  technology  can  provide  engineers  and  scientists  direct  access  to  graphical  super 
computing  capabilities  and  resources  while  minimizing  lengthy  and  redundant  data  transfer 
times,  costly  licenses,  and  the  inconvenience  of  travel. 
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Introduction 

A  persistent  challenge  in  the  High-Performance  Computing  (HPC)  community  is 
how  to  provide  remote  visualization  capability  to  its  users.  The  RDECOM-TARDEC 
HPC  Group  has  addressed  this  problem  locally  within  its  installation  through  a  legacy 
system  -  an  elaborate  network  of  Lightwave  VDE/200  KVM-over-Fiber  (Keyboard, 
Video  and  Mouse)  devices  installed  throughout  the  TARDEC  campus.  Implementation 
of  this  system  required  the  deployment  of  dedicated  multi-mode  fiber  optic  lines  to  serve 
as  the  transmission  medium.  This  solution,  while  effective  within  the  boundaries  of 
TARDEC,  fails  to  address  the  issue  of  long-distance  transmissions  of  KVM  data,  as 
multi-mode  fiber  can  only  span  about  two  miles  before  signal  degradation  occurs. 
Although  single-mode  fiber  KVM-over-Fiber  devices  exist  that  are  capable  of  longer 
distance  transmissions,  the  high  cost  of  these  devices  coupled  with  the  difficulties — and 
cost — of  installing  dedicated  long-distance  fiber  optic  lines  between  installations  make 
this  an  impractical  solution  for  widespread  deployment.  Aggravating  the  situation  further 
is  the  fact  that  dedicated  fiber  is  best  suited  for  installation  for  point-to-point 
communications. 

A  more  dynamic  and  economical  solution  is  a  KVM-over-IP  technology,  which 
uses  a  pre-existing  TCP/IP  network  to  transmit  KVM  data  between  two  locations. 
However,  the  level  of  perfonnance  and  functionality  present  in  the  current  consumer 
KVM-over-IP  devices  makes  them  less  than  desirable  for  DoD  High-Performance 
Computing  applications. 

To  address  the  specific  needs  of  the  DoD  HPC  community,  the  TARDEC  HPC 
Group  undertook  a  three-year  development  effort  through  the  pursuit  of  an  Army-funded 
Phase-II  Small  Business  Innovative  Research  (SBIR)  effort  with  IP  Video  Systems 
(fonnerly  known  as  TeraBurst)  to  produce  an  enhanced  version  of  their  V2D  product 
(henceforth  referred  to  as  “V2D-SBIR”).  At  the  time  of  this  paper’s  publication,  the 
SBIR  will  be  near  completion.  Additional  testing  and  future  demonstrations  are  expected 
subsequent  to  this  paper. 

Functionality 

To  accommodate  remote  use  of  the  high-end  visualization  capabilities  of  a  DoD 
High-Perfonnance  Computing  facility,  many  advanced  features  are  necessary. 
TARDEC-HPC's  SBIR  with  IP  Video  Systems  indicated  specific  requirements  for 
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creating  a  KVM-over-IP  device  that  could  be  used  for  HPC  visualization  purposes. 

These  requirements  included  support  for  a  USB  keyboard  and  mouse,  multi-channel 
digital  audio,  full-duplex  RS232  transmissions,  and  receiver-side  graphic  genlock 
support. 

A  significant  requirement  for  a  modern  KVM-over-IP  device  is  USB  keyboard 
and  mouse  support  over  the  USB  Human  Interface  Device  standard.  At  the  time  of  the 
SBIR’s  initial  signing,  few  consumer-level  KVM-over-IP  products  supported  the  use  of 
USB  devices,  instead  preferring  the  aging  PS/2  interface.  To  address  this,  adding  a  USB 
keyboard  and  mouse  control  was  a  specific  requirement  in  the  SBIR  contract,  and 
TARDEC  engineers  have  verified  that  this  functionality  now  exists  within  the  V2D- 
SBIR.  Incorporating  a  generalized  USB  subsystem  that  would  support  the  transmission 
of  any  USB  signal  was  beyond  the  scope  of  this  development  effort.  Therefore,  the 
SBIR-enhanced  V2D  is  capable  of  transmitting  USB  keyboard  and  mouse  signals  over 
IP,  but  it  does  not  support  USB  motion  trackers  or  other  USB  devices  at  this  time.  This 
capability  may  be  addressed  in  a  later  version  of  the  V2D-SBIR  system. 

Another  requirement  for  widespread  KVM-over-IP  use  within  HPC  facilities  is 
support  for  multi-channel  digital  audio.  This  is  especially  necessary  for  supporting 
visualization  systems  such  as  CAVE-like  environments  that  utilize  3D  spatial  audio. 

High  perfonnance  computing  visualization  environments  typically  use  AD  AT  audio 
signals  for  transmitting  digital  audio  from  supercomputers.  Therefore,  AD  AT  audio 
support  with  a  minimum  of  four  channels  was  included  as  a  requirement  for  the  SBIR 
effort.  TARDEC  engineers  have  verified  that  the  device  can  transmit  at  least  one  channel 
of  audio  over  the  AD  AT  port,  and  a  full  four-channel  test  will  be  conducted  in  the  near 
future. 

Visualization  environments  such  as  CAVE-like  displays  use  a  variety  of  motion 
tracking  devices  for  user  interaction.  The  majority  of  these  devices  use  an  RS232  serial 
interface,  making  it  a  necessity  for  virtual  reality  and  visualization  capabilities.  To  create 
a  KVM-over-IP  system  tailored  for  HPC  visualization  tasks,  TARDEC  included  RS232 
support  as  a  requirement  in  the  SBIR  effort.  As  a  result,  the  V2D  is  now  capable  of  full- 
duplex  transmission  of  serial  data  over  the  RS232  port.  The  functionality  has  been 
verified  by  IP  Video  Systems  by  performing  a  serial  file  transfer  between  two  computers 
using  the  serial  subsystem  of  the  V2D-SBIR. 

Unlike  most  users  of  a  KVM-over-IP  system,  a  major  need  for  DoD  HPC  users  is 
the  ability  to  synchronize  the  output  of  multiple  video  displays.  Since  practically  all 
CAVE-like  displays  use  multiple  video  outputs  to  produce  a  composite  image,  a  means  to 
synchronize  the  screen  refresh  rates  through  support  of  genlock  is  necessary  to  produce  a 
correct  immersive  3D  environment  using  active  stereo.  IP  Video  Systems  was 
successful  in  incorporating  this  capability  into  the  V2D-SBIR  device.  Through  internal 
tests,  they  have  successfully  synchronized  six  monitors  with  the  V2D  developed  under 
this  SBIR.  TARDEC  experiments  with  the  V2D-SBIR’s  genlock  system  will  occur  once 
TARDEC  receives  all  of  the  deliverables  upon  completion  of  the  SBIR. 

Performance 

In  order  to  transmit  video  over  IP  at  an  acceptable  perfonnance  level,  the  V2D- 
SBIR  applies  lossy  compression  to  the  signal.  To  achieve  this,  the  V2D-SBIR  device 
features  a  field  programmable  gate  array  (FPGA)  chip  to  implement  their  proprietary 
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compression  algorithms  in  hardware,  which  allows  for  very  high-speed  compression  of 
video  data.  Lossy  compression  implies  degradation  in  image  quality,  but  due  to  the 
limited  bandwidth  of  most  conventional  TCP/IP  networks,  a  compromise  must  be  made 
between  image  quality  and  frame  rate.  Given  the  subjective  nature  of  image  quality,  this 
compromise  can  only  be  made  by  the  end  user.  Despite  the  burden  for  the  user  to 
determine  the  compression  and  speed  balance  needed  for  their  specific  application,  most 
KVM-over-IP  devices  on  the  market  do  not  provide  options  for  configuring  the  amount 
of  compression  being  applied  to  the  data  stream.  In  contrast,  the  V2D-SBIR  offers  the 
user  an  extensive  level  of  configuration  options  for  finding  a  perfect  balance. 

The  V2D-SBIR  hardware  differentiates  between  lossy  compression  performed  on 
parts  of  the  display  image  that  stay  mostly  static  (such  as  large  parts  of  a  typical  desktop 
in  a  GUI)  and  on  parts  of  the  display  image  that  change  rapidly  (such  as  full-motion 
video).  The  amount  of  lossy  image  compression  performed  on  the  mostly  static  areas  of 
the  screen  is  detennined  by  a  user-defined  coefficient  labeled  “Low  Compression”,  which 
is  represented  as  an  integer  between  zero  and  ten.  Likewise,  the  amount  of  image 
compression  performed  on  the  more  dynamic  areas  of  a  display  image  is  labeled  “High 
Compression”,  and  its  intensity  is  also  represented  by  an  integer  coefficient  between  zero 
and  ten. 

TARDEC  HPC  engineers  have  developed  several  in-house  automated 
benchmarking  tools  that  allow  the  team  to  measure  the  performance  of  the  V2D-SBIR 
hardware.  One  of  these  tools  systematically  sets  the  hardware  to  all  121  combinations  of 
Low  and  High  Compression  for  five  seconds  each  and  measures  the  average  bandwidth 
usage  for  that  duration.  A  full  table  of  these  bandwidth  values  can  be  created  in  just  over 
ten  minutes,  during  which  a  consistent  source  of  animation  is  continuously  sent  over  the 
device.  The  frame  rate  of  the  animation  is  ideally  locked  at  its  maximum  value  (the 
refresh  rate  of  the  video  signal).  If  there  is  insufficient  bandwidth,  the  remote  display 
frame  rate  will  suffer,  resulting  in  undesirable  graphic  artifacting. 

In  obtaining  the  data  contained  in  tables  1  and  2,  the  V2D-SBIR  hardware  had 
100  megabits  of  bandwidth  available  over  the  internal  TARDEC  network,  which  is  the 
maximum  speed  of  the  V2D’s  network  card.  Therefore,  if  the  device  is  using  less 
bandwidth  than  the  maximum  (the  maximum  was  considered  to  be  85  Mb/sec  in  this  test 
due  to  transmission  overhead),  it  is  assumed  that  the  video  is  displaying  at  full  speed. 

For  this  paper,  two  tests  with  two  different  video  inputs  were  perfonned.  One 
video  input  was  a  five-second  excerpt  from  the  720p  BBC  Motion  Gallery’s  “Andes” 
video  available  on  Apple.com  running  in  a  continuous  loop  (to  synchronize  with  the 
bandwidth  sampling  period),  which  was  scaled  and  letterboxed  to  1024x768  at  60Hz. 

This  was  chosen  to  be  a  realistic  demonstration  of  real-time  video  streaming  for  the  V2D- 
SBIR  hardware.  The  other  video  input  used  was  the  Windows  XP  “Beziers”  screensaver 
at  1024x768  at  60Hz  with  all  settings  at  maximum,  which  was  determined  to  be 
extremely  demanding  of  the  V2D’s  compression  system.  This  was  chosen  to  be  a 
demonstration  of  a  worst-case  yet  realistic  scenario  that  the  V2D  may  have  to  encounter 
in  regular  use. 

Tables  1  and  2  provide  a  summary  of  the  amount  of  bandwidth  necessary  to  send 
two  different  types  of  video,  each  with  a  different  level  of  general  information  entropy,  at 
every  combination  of  compression  settings  offered  by  the  V2D-SBIR  hardware, 
assuming  a  constant  frame  rate.  For  both  of  these  tests,  the  benchmarking  program  was 
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executed  three  times  and  the  results  were  averaged  into  one  table.  Note  that  there  are,  in 
fact,  settings  available  that  will  allow  for  a  very  usable  interactive  session  at  fairly  low 
bandwidths.  The  following  is  the  color  code  legend  for  the  tables.  (Refer  to  the  full- 
color  tables  at  the  end  of  this  paper). 


Color-Coding  Legend 

Green 

Can  be  sent  over  a  DS3  (<45  Mb/sec) 

Yellow 

Can  be  sent  over  LAN  (Between  45  Mb/sec 
and  80  Mb/sec) 

Red 

Cannot  be  sent.  (>80  Mb/sec) 

Networking 

The  intent  behind  the  V2D-SBIR  project  is  that  the  device  can  transfer  data  from 
the  transmitter  to  the  receiver  over  any  IP-based  network.  However,  due  to  security  and 
performance  practicalities  regarding  conventional  networks  (firewalls,  NATs,  QoS 
issues,  etc),  one  must  also  take  into  account  the  network  that  the  V2D-SBIR  will  be 
transmitting  data  through. 

As  an  example  of  these  complications,  this  paper  will  cover  the  basic  network 
issues  encountered  during  TARDEC-HPC’s  first  Internet  test  of  the  V2D-SBIR  which 
was  conducted  between  the  Detroit  Arsenal  in  Warren,  MI,  and  its  partner,  Automation 
Alley,  headquartered  in  Troy,  MI  (approximately  nine  miles  away).  As  TARDEC  and 
Automation  Alley  are  two  separate  organizations,  their  networks  are  independent, 
connected  to  each  other  only  via  the  Internet.  The  relatively- short  distance  between 
installations  and  the  network  separation  made  the  Automation  Alley  headquarters  an 
excellent  choice  for  our  first  Internet  test  of  the  V2D-SBIR.  Figure  1  depicts  a  logical 
network  diagram  used  for  these  tests. 

One  major  complication  in  the  use  of  a  KVM-over-IP  device  is  the  firewall 
protection  between  Internet  networks.  The  TARDEC/ Automation  Alley  test  had  many 
coordination  and  configuration  issues  with  firewalls  before  the  system  could 
communicate  reliably.  On  both  networks,  network  administrators  had  to  forward  TCP 
and  UDP  traffic  for  certain  ports.  Making  the  problem  even  worse  were  port  blocks 
imposed  by  network  authorities  above  the  TARDEC  command  level,  which  caused  traffic 
to  be  blocked  even  when  the  ports  were  supposedly  forwarded  locally.  After  realizing 
the  higher-level  issue,  TARDEC-HPC  successfully  found  an  authorized  port  that  worked 
between  the  two  networks,  which  allowed  for  data  transmission  between  the  two 
facilities. 

Another  complication  in  the  TARDEC/Automation  Alley  test  was  bandwidth 
limitations.  The  Automation  Alley  network  provided  an  Internet  connection  that  was 
able  to  communicate  consistently  at  only  2.5  megabits  per  second.  This  led  to 
significantly  reduced  frame  rates,  especially  at  a  resolution  of  1024x768.  However,  the 
speed  of  the  transmission  is  still  fast  enough  to  support  typical  desktop  usage,  although 
full  motion  video  at  2.5  Mbps  requires  extremely  high  compression  levels  to  obtain  a 
usable  frame  rate  of  over  10  frames  per  second. 

When  considering  the  use  of  KVM  over  IP  technologies,  another  networking 
issue  that  must  be  addressed  is  that  of  latency  and  packet  loss.  During  the 
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TARDEC/Automation  Alley  test,  there  was  relatively  little  network  latency,  and  packet 
loss  was  only  an  occasional  problem.  However,  the  hardware  is  capable  of  working  with 
even  a  one-second  latency,  but  would  make  any  interactive  use  of  the  V2D  effectively 
impossible.  Therefore,  obtaining  favorable  latency  and  low  packet  loss  between  the 
target  networks  is  a  prerequisite  for  proper  V2D-SBIR  operation. 

In  terms  of  security,  the  V2D-SBIR  offers  SSH  access  to  its  configuration  menu 
system  for  secure  login,  as  well  as  a  serial  console  and  a  direct  monitor/keyboard 
connection.  The  V2D-SBIR  does  not  offer  on-board  encryption  beyond  its  compression 
algorithms  for  the  video  feed  or  the  keyboard/mouse/audio  information.  In  order  to 
facilitate  a  secure  transmission  of  V2D-SBIR  data  over  the  Internet,  IP  Video  Systems 
recommends  the  use  of  dedicated  hardware  VPN  devices  that  can  transmit  over  UDP. 

Although  KVM-over-IP  devices  are  connected  via  IP  networks,  problems 
encountered  during  the  Automation  Alley  test  were  relatively  minor.  Though  when 
facing  a  larger  deployment  of  V2D-SBIR  hardware,  especially  for  a  site  that  may  have  a 
centralized  bank  of  transmitters,  the  coordination  of  port  settings  and  IP  address 
assignment  would  become  essential. 

Conclusion 

Overall,  the  V2D-SBIR  project  has  led  to  great  advances  in  KVM-over-IP 
technology,  particularly  in  the  realm  of  DoD  High  Perfonnance  Computing  expectations. 
However,  just  as  technology  continues  to  progress,  there  will  always  be  room  for 
improvement  in  the  KVM-over-IP  capabilities. 

One  improvement  TARDEC-HPC  would  like  to  see  is  a  generalized  USB 
transmission  system  capable  of  transmitting  any  USB  data  over  the  network  rather  than 
just  the  USB  keyboard-and-mouse-only  configuration  that  is  on  the  current  V2D-SBIR 
prototype.  Due  to  the  extremely  high  bandwidth  requirements  of  many  USB  devices,  it 
may  be  impractical  to  create  such  a  system  with  the  current  levels  of  bandwidth  available. 

Another  desired  improvement  is  Gigabit  Ethernet  support.  Although  only  the 
expensive,  high-bandwidth  Internet  connections  are  currently  capable  of  transmitting 
more  than  100  megabits  per  second  of  data,  preparing  for  the  future  of  extremely  high- 
bandwidth  Internet  connections  is  a  wise  decision.  As  KVM-over-IP  devices  are 
bandwidth-intensive  devices,  this  progression  would  allow  for  much  higher  resolution 
video  at  steadier  frame  rates  over  the  proper  connections. 

Another  possibility  for  additional  research  is  the  concept  of  a  small  form-factor 
version  of  a  KVM-over-IP  device,  specifically  one  that  could  run  within  the  PCI  bus  of  a 
workstation.  Such  a  system  would  allow  a  wide  variety  of  computers  to  feature  full 
KVM-over-IP  support  without  the  installation  of  rack-mount  V2D-SBIR  devices.  This 
step  would  require  additional  funding  and  further  research,  as  the  small  form-factor 
version  would  be  such  a  radical  change  over  the  rack-mount  system  that  a  large  amount 
of  architecture  redesign  would  be  necessary. 

Regardless  of  the  desired  improvements,  the  IP  Video  Systems  V2D-SBIR 
prototype  exposes  DoD  HPC  engineers  to  a  new  horizon  of  KVM-over-IP  transmission 
of  visualization  and  virtual  reality  environment  data.  As  the  cost  of  Internet  bandwidth 
goes  down  and  the  cost  of  dedicated  fiber  goes  up,  the  KVM-over-IP  solution,  with  its 
more  dynamic  connection  methodology,  will  have  a  definite  place  within  the  DoD  High 
Performance  Computing  community. 
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The  TARDEC  HPC  Group  and  IP  Video  Systems  are  looking  for  potential  DoD 
HPC  users  that  could  benefit  from  this  remote  visualization  capability.  We  can  provide 
the  hardware  and  apply  our  testing  experience  to  demonstrate  that  the  capability  does 
exist  and  meets  the  needs  of  the  DoD  HPC  community.  We  also  are  looking  for 
opportunities  and  customers  that  would  support  us  in  continuing  the  development  of  this 
capability  by  means  of  a  SBIR  Phase  III.  Contact  the  TARDEC  HPC  Helpdesk  for 
further  details  at  hpc@tacom.army.mil 
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Bandwidth  used  in  Mb/sec  (assuming  maximum 
framerate) 
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Table  1:  BBC  Motion  Gallery  "Andes"  excerpt,  1024x768@60Hz. 


Bandwidth  used  in  Mb/sec  (assuming 
maximum  framerate) 

Low\High  0  1  2  3456789  10 


Table  2:  Windows  XP  "Beziers"  screensaver,  1024x768@60Hz. 
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Figure  1:  Logical  network  diagram  for  the  TARDEC/Automation  Alley  tests 


